Workshop on evaluating personal search

The first ECIR workshop on Evaluating Personal Search was held on 18th April 2011 in Dublin, Ireland. The workshop consisted of 6 oral paper presentations and several discussion sessions. This report presents an overview of the scope and contents of the workshop and outlines the major outcomes

Inferring User Knowledge Level from Eye Movement Patterns

The acquisition of information and the search interaction process is influenced strongly by a person’s use of their knowledge of the domain and the task. In this paper we show that a user’s level of domain knowledge can be inferred from their interactive search behaviors without considering the content of queries or documents. A technique is presented to model a user’s information acquisition process during search using only measurements of eye movement patterns. In a user study (n=40) of search in the domain of genomics, a representation of the participant’s domain knowledge was constructed using self-ratings of knowledge of genomics-related terms (n=409). Cognitive effort features associated with reading eye movement patterns were calculated for each reading instance during the search tasks. The results show correlations between the cognitive effort due to reading and an individual’s level of domain knowledge. We construct exploratory regression models that suggest it is possible to build models that can make predictions of the user’s level of knowledge based on real-time measurements of eye movement patterns during a task session

User behaviour and task characteristics: A field study of daily information behaviour

Previous studies investigating task based search often take the form of lab studies or large scale log analysis. In lab studies, users typically perform a designed task under a controlled environment, which may not reflect their natural behaviour. While log analysis allows the observation of users' natural search behaviour, often strong assumptions need to be made in order to associate the unobserved underlying user tasks with log signals. We describe a field study during which we log participants' daily search and browsing activities for 5 days, and users are asked to self-annotate their search logs with the tasks they conducted as well as to describe the task characteristics according to a conceptual task classification scheme. This provides u

Search strategies of Wikipedia readers

The quest for information is one of the most common activity of human beings. Despite the the impressive progress of search engines, not to miss the needed piece of information could be still very tough, as well as to acquire specific competences and knowledge by shaping and following the proper learning paths. Indeed, the need to find sensible paths in information networks is one of the biggest challenges of our societies and, to effectively address it, it is important to investigate the strategies adopted by human users to cope with the cognitive bottleneck of finding their way in a growing sea of information. Here we focus on the case of Wikipedia and investigate a recently released dataset about users’ click on the English Wikipedia, namely the English Wikipedia Clickstream. We perform a semantically charged analysis to uncover the general patterns followed by information seekers in the multi-dimensional space of Wikipedia topics/categories. We discover the existence of well defined strategies in which users tend to start from very general, i.e., semantically broad, pages and progressively narrow down the scope of their navigation, while keeping a growing semantic coherence. This is unlike strategies associated to tasks with predefined search goals, namely the case of the Wikispeedia game. In this case users first move from the ‘particular’ to the ‘universal’ before focusing down again to the required target. The clear picture offered here represents a very important stepping stone towards a better design of information networks and recommendation strategies, as well as the construction of radically new learning paths

Search as learning (SAL) workshop 2016

The "Search as Learning" (SAL) workshop is focused on an area within the information retrieval fi

What makes re-finding information difficult? A study of email re-finding

Re-nding information that has been seen or accessed before is a task which can be relatively straight-forward, but often it can be extremely challenging, time-consuming and frustrating. Little is known, however, about what makes one re-finding task harder or easier than another. We performed a user study to learn about the contextual factors that influence users' perception of task diculty in the context of re-finding email messages. 21 participants were issued re-nding tasks to perform on their own personal collections. The participants' responses to questions about the tasks combined with demographic data and collection statistics for the experimental population provide a rich basis to investigate the variables that can influence the perception of diculty. A logistic regression model was developed to examine the relationships be- tween variables and determine whether any factors were associated with perceived task diculty. The model reveals strong relationships between diculty and the time lapsed since a message was read, remembering when the sought-after email was sent, remembering other recipients of the email, the experience of the user and the user's ling strategy. We discuss what these findings mean for the design of re-nding interfaces and future re-finding research

User Behaviour and Task Characteristics: A Field Study of Daily Information Behaviour

Previous studies investigating task based search often take the form of lab studies or large scale log analysis. In lab studies, users typically perform a designed task under a controlled environment, which may not reflect their natural behaviour. While log analysis allows the observation of users' natural search behaviour, often strong assumptions need to be made in order to associate the unobserved underlying user tasks with log signals. We describe a field study during which we log participants' daily search and browsing activities for 5 days, and users are asked to self-annotate their search logs with the tasks they conducted as well as to describe the task characteristics according to a conceptual task classification scheme. This provides us with a more realistic and comprehensive view on how user tasks are associated with logged interactions than seen in previous log- or lab-based studies; and allows us to explore the complex interactions between task characteristics and their presence in naturalistic tasks which has not been studied previously. We find a higher number of queries, longer timespan, as well as more task switches than reported in previous log based studies; and 41% of our tasks are zero-query tasks implying that large amounts of user task activities remain unobserved when only focused on query logs. Further, tasks sharing similar descriptions can vary greatly in their characteristics, suggesting that when supporting users with their tasks, it is important to know not only the task they are engaged with but also the context of the user in the task

Navigating through digital folders uses the same brain structures as real world navigation

Efficient storage and retrieval of digital data is the focus of much commercial and academic attention. With personal computers, there are two main ways to retrieve files: hierarchical navigation and query-based search. In navigation, users move down their virtual folder hierarchy until they reach the folder in which the target item is stored. When searching, users first generate a query specifying some property of the target file (e.g., a word it contains), and then select the relevant file when the search engine returns a set of results. Despite advances in search technology, users prefer retrieving files using virtual folder navigation, rather than the more flexible query-based search. Using fMRI we provide an explanation for this phenomenon by demonstrating that folder navigation results in activation of the posterior limbic (including the retrosplenial cortex) and parahippocampal regions similar to that previously observed during real-world navigation in both animals and humans. In contrast, search activates the left inferior frontal gyrus, commonly observed in linguistic processing. We suggest that the preference for navigation may be due to the triggering of automatic object finding routines and lower dependence on linguistic processing. We conclude with suggestions for future computer systems design